Bias Detection: Study Identifies Instruments for Evaluating Animal Studies

No good scientist wants to produce, or be accused of producing, a poorly conducted study. Even so, toxicologists conducting animal studies are not widely expected to document the steps they take to ensure internal validity—that is, to prevent their results from being skewed due to a methodological issue (a concept known as risk of bias). This is likely to change soon, and a team of researchers at the University of California, San Francisco (UCSF) has taken a first step toward a solution by conducting a systematic literature review that identifies 30 instruments for evaluating risk of bias in animal research. 1 " Risk of bias " refers specifically to the introduction of systematic errors as a result of the way a study was conducted; it is unrelated to prejudice or manipulation of results to achieve a desired outcome. Other aspects of study quality include external validity (the extent to which results can be generalized) and reporting quality (descriptions of the experiment design, conduct, and analysis). 2 Tools to assess these other aspects of study quality are familiar to toxicologists, but the concept of risk of bias is relatively new in environmental health, says Kris Thayer, director of the National Toxicology Program's Office of Health Assessment and Translation (OHAT), who was not involved in the new review. Thayer's group is leading efforts to incorporate systematic review methodology into OHAT evaluations. 6 She says it's important, when conducting these evaluations, to clearly define which aspects of study quality are being considered and to present them as discrete elements. She lauds the new review for providing a starting point for determining how to evaluate the risk of bias in animal studies. The review presents an inventory of different approaches for assessing existing study-quality tools and summarizes what is known about specific factors as sources of systematic bias. " Although there is a well-developed and empirically based literature on how to evaluate the risk of bias of randomized controlled clinical trials, less is known about how to do this for animal studies, " wrote the team of UCSF researchers, who were led by Lisa Bero, an expert in systematic review methods and evidence-based health care. Bero and her coauthors combed through 45 years of MEDLINE publications and found 3,731 articles describing instruments for assessing risk of bias in animal studies. They identified 30 instruments that assess anywhere from 2 to 25 criteria associated with …